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Be it known that ERIN M. PANTTAJA, a U.S. Citizen residing in SOMERVILLE, 
MASSACHUSETTS has invented certain improvements in a METHOD AND SYSTEM 
FOR PRONOUN DISAMBIGUATION of which the following description in connection 
with the accompanying drawings is a specification, like reference characters on the drawings 
indicating like parts in the several figures. 



METHOD AND SYSTEM FOR PRONOUN DISAMBIGUATION 



CROSS-REFERENCE TO RELATED APPLICATIONS 
Not Applicable 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH 

Not Applicable 
REFERENCE TO MICROFICHE APPENDIX 

Not Applicable 
BACKGROUND OF THE INVENTION 

This invention relates to user interfaces which permit the use of pronouns 
and, more particularly, to a method and system for disambiguation of pronouns used in a 
user interface. 

In a voice user interface, such as a voice personal assistant with an address book, it is 
often convenient to allow the user to use pronouns. For example, after the user has listened to 
a message from John Jones, he should be able to say "call him back," in order to return the 
call. However, the use of pronouns introduces the difficulty that the entries (e.g. people and 
organizations) in the user's address book may be male, female, or corporations, such that the 
pronouns him, her, it, and them are possible inputs by the user in a given interaction. 

One method of processing input containing pronouns is to allow the user to use 
pronouns indiscriminately. In this method, all pronouns are allowed and equivalent in the 
grammar or user response and the system assumes that the user will use the pronoun that is 
appropriate. Even if the user does not use the right pronoun, the pronoun equivalency will 
probably result in the system performing the correct action (the action the user intended). 
This is the method most systems use today. 



However, in these prior art systems, the system is receiving (and the user is speaking) 
information that is essentially being disregarded. With the problems that exist with speech 
recognition today, even this small amount of information could help to improve recognition. 
It should also be noted that in some cases (common first names, some company names), it is 
easy to select a pronoun for a given name, but in other cases (the proverbial boy named Sue, 
or more common names like Jan, Jordan, Randy, Sandy and Shelly) it is quite likely to be 
incorrect. This means that if the system assigns a gender to each possible entry based on the 
noun identifier or name provider, the gender may be inappropriate. In this case, it is not just 
a grammatical issue; the system is more difficult to use, as it requires the user to learn an 
assigned "gender" which may have little basis in reality. 

Some programs allow the user to provide gender-indicating information, for example 
a title like Mr. or Mrs. However, not all programs allow (or require) users to specify a title 
and those that do typically also accept titles (e.g. Dr., Hon., or Gen.) that are gender-neutral. 
In addition, it is not uncommon for a user to be unaware of the gender of a person in their 
address book, particularly if all of their communication has been via email or other written 
correspondence. 

Accordingly, it is an object of the present invention to provide an improved user 
interface. 

It is another object of the present invention to provide a method and system for using 
pronouns in a user interface. 

It is a further object of the present invention to provide a method and system for 
disambiguating the use of pronoims in a user interface. 



SUMMARY OF THE INVENTION 

The present invention is directed to a method and system for disambiguating pronouns 
used by a user during an interaction with the system. The method and system according to 
the invention allow the user to input pronoun references to nouns or proper nouns used in the 
context of the interaction. The method and system associate pronoun references with nouns 
and proper nouns input by a user and learn or modify the associated pronoun references 
through experience based upon usage by a user. 

The system can include a user reference database (such as a user address book) which 
lists entries by one or more identifiers or names. The user reference database can include one 
or more fields or records for including pronoun reference information to be associated with 
each entry. Alternatively, the system can include a separate pronoun database which 
provides pronoun references associated with entries in a user reference database. When a 
user inputs a new entry into the reference database, the user can be prompted or required to 
input pronoun reference information (either directly "him" or "her" or indirectly Mr. or Mrs.). 
Alternatively (or in addition), the system can include a search component for searching 
entries in one or more databases for entries having the same as or similar identifier 
information to a given entry in the user reference database and assigning a pronoun 
designation as a function of the results of the search. The search component can be adapted 
to search any or all of the available databases, including the user reference database, a 
default pronoun database of common identifiers, a unified user reference database and/or a 
plurality of user reference databases. 

The system can also include a historical list of references to nouns and proper nouns 
used during the interaction between the user and the system. The historical list can be a list 
of all nouns to which the user and/or the system has referred within a given session. The list 
can be limited to nouns mentioned (or referred to via pronouns) within a given time period. 



number of utterances, number of nouns, or otiier constraint. Upon receiving input from a user 
containing a pronoun reference, the system can be adapted for using the pronoun information 
contained in one or more databases to determine the noun or proper noim that the pronoun 
could refer to. The system can be adapted to compare an input (or recognized) pronoun 
received by the system with a listing of pronouns corresponding to one or more of the nouns 
or proper nouns in the historical list, in order to properly interpret the user's input. The 
system can select the record in the user reference database corresponding to a noun or proper 
noun as a function of the order in the historical list (e.g. the most recent) and the pronoun 
preference associated therewith. The system can use the information contained in that record 
as needed to complete the task requested by the user. The pronouns corresponding to the 
nouns or proper noims in the historical list can be initially selected by the system as a 
function of 1) a predefined system database that associates identifiers of nouns and proper 
nouns; 2) the user reference database; 3) a unified user reference database and/or a other 
databases or sources of pronoun reference information. After the initial selection, the 
pronoun preference information can be modified as a fimction of usage. 

After the system has selected an entry corresponding to one of the most recently 
identified nouns or proper notms in the history list that matches the pronoun recognized by 
the system, the system can prompt the user to verify the entry selected by the system. Based 
upon the user response, the system can update or modify the pronoun information in one of 
the system databases to permit the system to learn or adapt based upon pronoun usage. For 
example, where the user verifies the correct selection, a value for the pronoun associated with 
noun or proper noun selected can be incremented or modified in recognition of the accepted 
usage or where the user indicates an incorrect selection, a value for the pronoun associated 
with noun or proper noun selected can be decremented or modified in recognition of the 
unaccepted usage. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing and other objects of this invention, the various features thereof, as well 
as the invention itself, may be more fully understood from the following description, when 
read together with the accompanying drawings in which: 

FIGURE 1 is a system for providing a voice based personal assistant in accordance 
with the present invention; 

FIGURE 2 is a diagrammatic view of an application processing unit in accordance 
with the present invention; 

FIGURES 3 and 4 show flow charts of processes for associating pronoun references 
with identifiers or names in a reference database in accordance with the present invention; 
and 

FIGURE 5 is a flow chart for interpreting pronoun references and learning pronoun 
preferences in accordance with the present invention. 
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention is directed to a system for providing a user interface which can 
accept and interpret input containing pronoims. The system can learn or modify pronoun 
preferences by storing pronoun preference information and by modifying that pronoun 
preference information based upon usage. The present invention is also directed to a method 
for interpreting input which contains pronouns using pronoun preference information. The 
method can include storing pronoun preference information and modifying the pronoun 
preference information based upon usage. In order to facilitate a better understanding, the 
invention is described below with respect to one or more illustiative embodiments from 
which a person having ordinary skill in the field can make and use the invention. 

FIGURE 1 shows a system 10 for providing a voice based user interface in 



accordance with the present invention. The system 10 is, in part, a conventional information 
services system as described in U.S. Patents 5,029,199; 5,263,080; 5,193,110; 5,659,599; 
5,524,139; and 5,402,472, all assigned to Comverse Network Systems, Inc. of Wakefield, 
Massachusetts and incorporated herein by reference. The System 10 is connected to a 
Central Office 12 into which users may call and include optional telephone network interface 
circuitry 14 including Chaimel Bank 16. The system 10 further includes one or more Master 
Control Units (MCU) 30 that control digital switching system 32 to route calls from the 
central office 12 to one or more of the application processing units (APU) 20. Each APU 20 
can include a computer having a central processing unit (CPU) 22 and data storage (HD) 40 
and telephone network termination, such as a Tl termination that can provide up to 24 voice 
ports or telephone interface ports 26. The APU 20 performs the user call processing and 
allows the user access to user and system data stored in data storage 40 or other system 
storage (not shown). 

FIGURE 2 shows a diagrammatic view of an application processing unit 20 in 
accordance with the present invention. The APU 20 can include a noun reference database 
42, a pronoun reference database 44 and a historical listing 46. The APU 20 can further 
include a voice assistant 52 that can facilitate user interaction with one or more voice 
applications 54. The voice assistant 52 can allow a user to interact with one or more voice 
applications 54 using spoken commands and numeric keypad input. 

For purposes of illustration, the present invention is described herein with respect to a 
voice personal assistant which allows a user to access a voice application, such as a voice 
messaging system. The voice messaging system can provide a noun reference database, for 
example, an address book which provides telephone numbers and/or email addresses 
associated with norms and proper nouns (such as individual people or organizations) by a 
unique noun identifier such as a name. The names can be entered into the address book by 



the user or the system can enter the names based upon external input, such as caller ID data 
or other caller data (including speech recognition of caller input data). The voice personal 
assistant 52 can track the nouns and proper nouns used or referenced by the user in the 
historical listing 46 which can include a chronological listing (or database) of names used by 
either the user or the voice personal assistant 52 or both within a session, time period or other 
constraint 

APU 20 can be an IBM Compatible computer based upon an Intel (Intel Corp., Santa 
Clara, Calif.) or compatible microprocessor and running a Microsoft Windows (Microsoft 
Corp., Redmond, Wash.), Unix based or Linux based operating system. The APU 20 can 
include RAM and non-volatile storage, such as a hard disk, and a network interface (e.g. 
Ethernet or Fast Ethernet - not shown). The network interconnects each of the APUs 20 and 
the MCUs 30 to allow information to be sent between them. The system can fiirther include a 
network storage device (not shown) which can allow one or more APUs 20 or MCUs 30 to 
store and access data from a common location. 

In accordance with the invention, the system can further provide a means for 
associating one or more pronoun preferences with one or more the entries in the address 
book. This can be accomplished, for example, by providing additional fields in the address 
book for pronoun information or by providing a separate pronoun database 44 which is linked 
to one or more entries in the address book. 

Table 1 provides an example of a pronoun database or log in accordance with the 
present invention. 



TABLE 1. 



Noun Identifier 


Him 


Her 


Them 


It 


User 


John 


4 








Tim Smith 


Pat 




3 


1 




User 1234 


American Airlines 


1 




10 


2 


Admin 


Weather 








6 


99-1234 



The pronoun database can include a listing of noun identifiers or names and one or more 
pronoun fields which keep a count of the number of times that the user referred to the noun 
(e.g. the person or organization) by a specific pronoun (e.g. him, her, them or it). The 
pronoun fields can be part of the user's address book or provided as part of a separate 
pronoun database. 

Where a separate pronoun database is provided, it is not necessary for the system to 
include a noun reference database, such as an address book. For example, where the system 
provides simple voice mailbox services, which does not provide a user address book, the 
system can create and maintain a pronoun database transparent to the user. The separate 
pronoun database can be specific to a given user or a global database common to all users. 
Where the separate pronoun database is common to all users, it can be linked, such as via a 
USER field to a specific user of the system. 

The structure of the pronoun database can be adapted based upon the needs of the 
system and the language of the system users. For example, a system which only references 
people might only track the masculine and feminine pronouns (e.g. him and her) and 
similarly, a system which only references organizations or business entities might only track 
singular and plural pronouns (e.g. it and them). A system which uses a non-English language 
can track other pronouns as can be appropriate (such as "ellos," the masculine third person 
plural pronoun in Spanish). In addition, a single field can be used to track mutually exclusive 
pronoun preferences, for example, a masculine/feminine pronoun field can track usage by its 



numeric value whereby, for example, a positive nximber is indicative of feminine preference 
and a negative number is indicative of a masculine preference and the value is adjusted by 
adding or subtracting adjustment units based upon usage. A zero value could invoke a 
default preference or can be designated not valid (i.e. adding 1 to -1 changes the value to +1 
or using odd number and an adjustment unit value of 2). Alternatively, field could range 
from 1 to 10 where 1 indicates a strong preference toward, for example, a singular pronoun 
and 10 indicates a strong preference for a plural pronoun. In addition, the field values can be 
limited to prevent them getting too large and overflowing the field limit. For example, the 
values can be limited to + 10 and -10 or in from 1 to 10 and the modification process can be 
adapted to increment the field up to the limit value whereby increments beyond the limit 
value are ignored. 

For example, the data in Table 1 could appear as in Table 2, where in the masculine- 
feminine (M/F) field, the field range is -10 to +10 and a negative value indicates a masculine 
preference and a positive value indicates a feminine preference and in the singular-plural 
(S/P) field, the range is 1 to 10 and a value closer to 1 indicates a singular preference and a 
value closer to 10 indicates a plural preference. 

TABLE 2 



Noun Identifier 


M/F 


S/P 


User 


John 


-4 




Tim Smith 


Pat 


3 


6 


Userl234 


American Airlines 


-1 


8 


Admin 


Weather 




1 


99-1234 



FIGURE 3 shows a flow chart of a process 100 for creating a pronoun database in 
accordance with the present invention. In the process described, the system includes a 
separate pronoun database that is linked to the user reference database (the address book in 
our illustrative example). The process 100 starts at step 110 where a record is added to the 



noun reference database. A user can add the record, a new entry in their address book, or the 
system can add the entry upon receipt of an incoming call (with caller ID services or call 
input information) or receipt of a voice message or other external conamunication from which 
the necessary information can be derived. When a record is added to the user's address book, 
a corresponding entry can be added in the user's pronoun database in step 1 12. For example, 
if John Smith is added to the user's address book, an entry for John Smith is also added to the 
user's pronoun database. At step 1 14, the system searches the noun reference database for 
pronoun information. This includes searching the newly created entry in the noun reference 
database for gender information. If the noun reference database includes one or more gender 
specific fields in step 116, the information from those fields can be used to determine a 
pronoun preference and modify the entry in the pronoun database in step 118. For example, 
in the user address book the system can search for titles such as Mr. and Mrs. or even a 
specific male or female designation in the newly added record. The system can also search 
other records in the noun reference database for matching or similar entries. Where the noun 
references match in step 116, the information can be used to modify the entry in the pronoun 
database in step 118. For example, the given name John may match other address book 
entries and the pronoun information (masculine) associated with the other entries in the 
pronoun database can be used to formulate the pronoun preference for the newly created 
entry in the pronoun database. Optionally, the system can be configured to search other 
system or external databases in addition to or instead of searching the noun reference 
database. For example, if no pronoun information is found in step 1 16, the system can 
continue the search for more pronoun information in step 120 (and if no information is found, 
leave a predefined entry, such as a blank entry or a default value). 

In step 120, the process can continue to search other system databases for pronoun 
information. If pronoun information is found in step 122, the information can be used to 

10 



modify the entry in the pronoun database in step 124. The other system databases can 
include other user reference databases and user pronoun databases, a default system pronoun 
database which includes predefined default values for many possible name entries. If no 
pronoun information is found at step 122, the system can optionally proceed to step 126 in 

5 order to search external databases. 

In step 126, the process can continue to search external databases for pronoun 
information. If pronoun information is found in step 128, the information can be used to 
modify the entry in the pronoun database in step 130. External databases can include 
searching the Internet or establishing a connection, for example over a network, with a 

10 remote system that provides pronoun information. If no pronoun information is found at step 
128, the process ends. If no pronoun information is found at this (or any other) point, the 
process can include a step (not shown) whereby a default pronoun preference, such as her or 
it is used. 

It should be appreciated that it is not necessary for the system to include a noun 
15 reference database, such as a user address book. Where the user receives a voice message 
from caller, the system implementing the process above in accordance with the invention can 
add an entry for the caller in a pronoun database of the user based upon the available 
information. The system can use, for example, caller ID to determine the caller's telephone 
number and use the telephone number to retrieve the caller's name or prompt the caller for 
20 any needed information. 

FIGURE 4 shows a flow chart of a process 200 for creating a pronoun database in 
accordance with the present invention. In the process described, the pronoun data is included 
as part of the user reference database (the address book in our illustrative example). The 
process 200 starts at step 210 where a record is added to the noun reference database. A user 
25 can add the record, a new entry in their address book or the system can add the entry upon 

11 



receipt of an incoming call (with call ID services or call input information) or receipt of a 
voice message or other external communication from which the necessary information can be 
derived. When a record is added to the user's address book, default information can be 
added in the pronoun data fields of the address book in step 212. For example, if John Smith 
is added to the user's address book, an entry for "him" is also added to the user's address 
book. This entry can be purely arbitrary, "it" for all entries or the information can be 
determined as a function of additional information obtained in the succeeding steps. At step 
214, the system searches the noun reference database for pronoun information. This includes 
searching the newly created entry in the noun reference database for gender information. If 
the noun reference database includes one or more gender specific fields in step 216, the 
information from those fields can be used to determine a pronoun preference and modify the 
pronoun information in the noun reference database in step 218. For example, in the user 
address book the system can search for titles such as Mr. and Mrs. or even a specific male or 
female designation in the newly added record. The system can also search other records in 
the noun reference database for matching or similar entries. Where the noun references 
match in step 216, the information can be used to modify the pronoun information in the 
noun reference database in step 218. For example, the given name John may match other 
address book entries and the pronoun information (masculine) associated with the other 
entries in the pronoun database can be used to formulate the pronoun preference for the 
newly created entry in the user address book. Optionally, the system can be configured to 
search other system and external databases in addition to or instead of searching the noun 
reference database. For example, if no pronoun information is found in step 216, the system 
can continue the search for more pronoun information in step 220 (and if no information is 
found, leave a default value). 

In step 220, the process can continue to search other system databases for pronoun 
12 



information. If pronoun information is found in step 222, the information can be used to 
modify the pronoun information in the noun reference database in step 224. The other system 
databases can include other user reference databases and user pronoun databases, a default 
system pronoun database which includes predefined default values for many possible name 

5 entries. If no pronoun information is found at step 222, the process can optionally continue to 
step 226 in order to search external databases. 

In step 226, the process can continue to search external databases for pronoun 
information. If pronoun information is found in step 228, the pronoun information can be 
used to modify the pronoun information in the noun reference database in step 230. External 

10 databases can include searching the Internet or establishing a connection, for example over a 
network, with a remote system that provides pronoun information. If no pronoun information 
is found at step 228, the process ends. If no pronoun information is found at this (or any 
other) point, the process can include a step (not shown) whereby a default pronoun 
preference, such as her or it is used. 

15 In accordance with system and method of the present invention, the pronoun 

information obtained from the various sources searched can be weighted. Thus, for example, 
where a record for Pat indicates a gender specific title such as Mr., the information can be 
weighted higher than another record which does not include a gender specific title and only 
indicates that the first name "Pat" uses a feminine pronoun. Specific examples include: 

20 Gender specific fields in the user address book (Mr., Ms., Mrs., etc.) can be weighted atlO 
points; Object specific fields in the user address book (e.g. a "Company name") can be 
weighted atlO points as "them"; Entries added to the user's address book using a specific 
pronoun (e.g. "add him to my address book") can be weighted at 1 point (this low weight 
value is due to the possibility of the user or the recognition being incorrect); The most 

25 common gender for the same first name in the user's address book, all address books in the 



system, or an external database can be weighted at 5 points; The most common pronoun for 
the same first name in all address books in the system can be wei^ted at 10 points; The 
most common gender for the same first name from a country specific (to the person) database 
or shared system address book can be weighted at 10 points; A database of known 
5 companies (firom, for example, a stock market or other company listing) can be weighted at 
10 points. 

FIGURE 5 shows a flow chart of a process 300 for interpreting pronoun references 
and learning pronoun preferences in accordance with the present invention. The process 300 
begins at step 310 wherein the user provides input that includes a pronoun and the pronoun is 

10 recognized by the system at step 312. Upon recognition of the pronoun in step 312, the 

system examines the history list in step 3 14 in order to interpret the pronoun. If there are no 
entries in the history list at step 3 1 6, the system indicates to the user "I am not sure who 
him/her/them/it refers to, please enter a name" in step 3 1 8 and returns to step 3 10 for more 
input from the user. If there are many entries in the history list in step 320, the pronoun 

15 preference for the most recently used entry in the history list can be determined in step 322 
and compared with the pronoun recognized in step 324. If the pronouns match in step 324, 
the user can be prompted whether the correct entry was selected in step 328. If the user 
approves the selection in step 330, the pronoun information for that selection is modified as a 
result of the successful interpretation of the pronoun at step 332. The process proceeds to 

20 step 336 where the selected entry is used to perform a task, such as for example, place a 
telephone call or send a voice mail or e-mail message. 

If the pronoun preference for the most recently used entry in the history list does not 
match the pronoun recognized, the process enters a loop which determines and compares the 
pronoun preference for each entry in the history list, fi-om most recent to least recent, until the 

25 pronoun determined matches the pronoun recognized or some process defined limit is 



reached (for example, after considering the 5 or 10 most recent entries in the history list). 
After a match is found, flie process proceeds to step 328 to verify the selection and, if 
verified, modify the pronoun information and perform the requested task. 

If there is only one entry in the history list at step 320, the process continues to step 
334 where the selected entry is used to perform a task, such as for example, place a telephone 
call, send a voice mail or e-mail message. 

The sample dialog below provides fiirther illustration and understanding of the 
present invention: 
USER: Play my messages 

You have a message from John Jones... "Please talk to Pat Smith today about 
the project; it's due on Friday." 
Call Pat Smith. 

Calling Pat Smith. <ring> There is no answer from Pat Smith. Please record a 
voice message. <beep> 
USER: Hi, Pat, do you have time to talk at 3? I'll be in all morning. 

SYSTEM: Message sent to Pat Smith. 
USER: Call him back. 



10 SYSTEM: 



USER: 
SYSTEM 



In this example, "Call him back" could either indicate that the user wants to call John, 
20 for example, to tell him that he has tried to schedule a meeting with Pat, or that the user wants 
to call Pat, for example, because he has just realized that he already has a three o'clock 
meeting and needs to send another message to Pat Smitii to change the meeting time. 

If this dialogue happens, and the pronoun data in database is as shown in Table 3, Pat 
is usually referred to with a female pronoun. Thus the system will assume that the user 
25 means John, and will say "Calling John Jones, if this is wrong, say cancel." If the user says 

15 



cancel, nothing is recorded in the pronoun log. If not, this is counted as a successful use of 
the male pronoun for John and the pronoun information in the Him field of John is increased 
to 5. 

TABLE 3. 



Name 


Him 


Her 


Them 


It 


John 


4 








Pat 




3 


1 





If, on the other hand, the pronoun log is as described in Table 4, the hypothesis will be 
that "him" refers to Pat, as Pat is the most-recently-mentioned person for whom the 
preference could be male. 

TABLE 4. 



Name 


Him 


Her 


Them 


It 


John 


4 








Pat 


2 


1 


1 





In another embodiment, the pronoun database entries can consist of a name from the 
address book and an ordered linked list of symbols representing the pronouns in the historical 
order used in the system by the user as shown in Table 5. When a new entry is added to the 
address book, tiie system compiles all available pronoun information and adds the appropriate 
pronoun to the end of the linked list for each piece of pronoun information considered. 
Multiple copies of a pronoun can be added if the pronoun information has a high weighting, 
for example, if the pronoun information is rated as weighted at ten points in the previous 
example. If at any point the linked list has more than N members (for example, nine in this 
instantiation), the first element or the oldest element can be dropped. 
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TABLE 5. 



Name 


Pronoun Usage 


John 


him, him, him, him. 


Pat 


him, her, him, him 


Chris 


him, him, him, him, her, him, her, her 



When the user speaks a pronoun, the system will go through each of the possible 
entries in the history list and count the number of each pronoun in the linked list associated 
with each entry. If there is a clear majority, then that pronoun will be associated with the 
name. The pronouns in the linked list can be weighted based upon their age, such that the 
most recently entered pronoun on the list may be worth three times the oldest entry on the 
linked list (assuming that the user will eventually meet the person or otherwise determine the 
correct gender of the person in their address book). In addition, multiple pronouns can be 
associated with a particular name. 

One advantage of this embodiment is that it allows the system to be responsive to the 
user's vocabulary choice. If, for instance, the user finds that she has been using the wrong 
pronoun for a contact, the system will quickly learn the new usage with no additional work 
on the part of the user. 

The invention may be embodied in other specific forms without departing from the 
spirit or essential characteristics thereof. The present embodiments are therefore to be 
considered in respects as illustrative and not restrictive, the scope of the invention being 
indicated by the appended claims rather than by the foregoing description, and all changes 
which come within the meaning and range of the equivalency of the claims are therefore 
intended to be embraced tiierein. 



